Extended compartmental model for modeling COVID-19 epidemic in Slovenia (preprint)

In the absence of a systematic approach to epidemiological modeling in Slovenia, various isolated mathematical epidemiological models emerged shortly after the outbreak of the COVID-19 epidemic. We present an epidemiological model adapted to the COVID-19 situation in Slovenia. The standard SEIR model was extended to distinguish between age groups, symptomatic or asymptomatic disease progression, and vaccinated or unvaccinated populations. Evaluation of the model forecasts for 2021 showed the expected behavior of epidemiological modeling: our model adequately predicts the situation up to 4 weeks in advance; the changes in epidemiologic dynamics due to the emergence of a new viral variant in the population or the introduction of new interventions cannot be predicted by the model, but when the new situation is incorporated into the model, the forecasts are again reliable. Comparison with ensemble forecasts for 2022 within the European Covid-19 Forecast Hub showed better performance of our model, which can be explained by a model architecture better adapted to the situation in Slovenia, in particular a refined structure for vaccination, and better parameter tuning enabled by the more comprehensive data for Slovenia. Our model proved to be flexible, agile, and, despite the limitations of its compartmental structure, heterogeneous enough to provide reasonable and prompt short-term forecasts and possible scenarios for various public health strategies. The model has been fully operational on a daily basis since April 2020, served as one of the models for decision-making during the COVID-19 epidemic in Slovenia, and is part of the European Covid-19 Forecast Hub.

Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations (preprint)

Background: Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022. Methods: We used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported from a standardised source over the next one to four weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models past predictive performance. Results: Over 52 weeks we collected and combined up to 28 forecast models for 32 countries. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 84% of participating models forecasts of incident cases (with a total N=862), and 92% of participating models forecasts of deaths (N=746). Across a one to four week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over four weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models. Conclusions: Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than two weeks.